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PREFACE 


The  work  described  in  this  publication  was  performed  by  the 
Mathematical  Analysis  Research  Corporation  (MARC)  under  contract  to 
the  Jet  Propulsion  Laboratory,  an  operating  division  of  the  California 
Institute  of  Technology.  This  activity  is  sponsored  by  the  Jet 
Propulsion  Laboratory  under  contract  NAS7-918,  RE182,  A187  with  the 
National  Aeronautics  and  Space  Administration,  for  the  United  States 
Army  Intelligence  Center  and  School. 
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Results  of  Inappropriate  EEP  Normalization  Methods  In  Correlation 


INTRODUCTION 

We  assume  that  data  arrives  in  terms  of  ellipses:  the  location  of  the 
center  of  the  ellipse  and  the  length  and  orientation  of  axes  of  the  ellipse. 

Ellipses  are  to  be  95%  ellipses,  i.e.  the  probability  that  the 
specified  ellipse  contains  a  given  emitter  is  .95  (elliptical  error  probable 
•=  EEP  is  95%).  If  the  incoming  ellipse  data  are  not  for  a  95%  ellipse  but  are 
for  a  say  50%  ellipse  then  the  incoming  data  must  be  transformed.  This 
transformation  of  incoming  data  only  affects  the  length  of  the  axes  (not  the 
center  of  the  ellipse  or  orientation.)  If  the  transformation  is  from  a  50%  to 
a  95%  ellipse  then  the  axes  of  the  Incoming  ellipse  are  lengthened. 


The  inner  most  ellipse  is  a  50%  ellipse. 


o  n 


Tr.e  er’ror  is  too  frequently  stating  that  the  incoming  data  do  not  come 
f  :"om  a  specific  emitter  when  in  fact  they  do. 

I  I'nder  2) 

Tr.e"e  are  two  possible  errors. 

The  resultant  ellipse  being  too  small. 

Tr.e  location  of  the  center  of  the  resultant  ellipse  being  overly 
affected  by  data  based  on  small  sample  sizes. 

i  ITMRAL 

Tr.e  F  distribution  is  used  in  determination  of  EEPs  (elliptical  error 
rotatle)  for  some  programs  such  as  Guardrail.  Ellipse  confidence  level 
onversicn  algorithms,  combination  algorithms  and  combination  testing 
algorithms  assum.e  that  EE.Ps  are  based  on  the  Chi-square  distribution.  This 
memo  is  not  concerned  with  all  the  problems  that  result  from  this  discrepancy 
for  these  three  types  of  algorithms.  It  is  concerned  with  the  direct  impact 
on  the  confidence  level  conversion  algorithm  and  with  the  implications  of  this 
impact  for  the  combination  and  testing  algorithms. 

when  the  original  distribution  underlying  EEPs  is  F,  converting  ellipse 
ccnfider.ee  coefficients  to  the  95*  level  is  an  approximation  that  is  only 
valid  for  'large'  sample  sizes.  This  memo  is  designed  to  illustrate  the 
rmpact  cf  using  this  approximation  with  small  sample  sizes.  The  ideas 
introcuced  ir.clude; 

i)  Tne  interpretation  of  the  conversion  of  confidence  level  as  a 
rescaling  of  the  size  of  the  confidence  ellipse, 

ii)  T.ne  factors  affecting  the  scaling  constant,  specifically, 

(a)  the  confidence  level  of  the  incoming  ellipse 
(c)  whether  the  original  distribution  is  Chi-square  or  F 
(c)  sample  size  (but  only  if  the  original  distribution  is  F) 

iii)  Tne  amount  of  error  in  ellipse  size  that  can  occur, 

iv)  what  the  difference  between  chi-square  and  F  is  supposed  to 
represent . 

v)  The  effect  of  scaling  errors  of  the  type  being  examined  here  on 
correlation  testing.  (And  questions  concerning  use  of  that  test 
when  F  is  the  basis  for  formation  of  the  ellipse.) 

Vi)  The  effect  of  the  scaling  errors  being  studied  here  on  the  point 
estimate  location  determined  by  the  combination  algorithm.  (And 
questions  concerning  use  of  this  algorithm  when  F  is  the  basis  for 
formation  of  the  ellipse.) 

vii)  The  effect  of  the  scaling  errors  being  studied  here  on  the  EEP 
size  of  the  resultant  ellipse  using  the  combination  algorithm. 

when  incoming  EEPs  are  based  on  sample  sizes  of  5  or  smaller  these 
considerations  are  very  important. 

Tne  concepts  outlined  above  will  be  discussed  in  the  sections  below. 

Tney  are  also  illustrated  in  the  graphs  that  follow,  and  may  be  pursued 
further  using  the  tables  attached  in  the  appendices.  (Other  graphs  concerning 
ellipse  combination  and  testing  for  combination  may  easily  be  imagined  in 
terms  of  the  geometric  characterization  of  the  testing  and  combination 
algor  i trims .  ) 


-i4- 


V. 


I 


r 


I.  IN’TF:.RPRi:TAT:or.'  of  CjNTIl'ENCE  level  CCN'VERSION  as  rescaling  EEP’3 

Changing  confidonoe  level  nay  be  thought  of  as  scaling  of  the  ECP  (size 
cf  a  confidence  ellipse.)  In  fact  given  two  confidence  levels  with  specified 
confidence  "ctr.cdclogi es  and  sarr.ple  size  (for  the  F)  it  is  possible  to  list 
the  scaling  factor.  For  example,  if  sample  size  is  5  (and  assuming  our 
apolications  have  2  spatial  degrees  of  freedom)  then  a  90?  EEP  based  on  the  F 
cisiribution  is  approximately  2.5  times  as  large  as  the  correspond i ng  50?  EEP. 
Tne  95?  EEP  approxi.mately  3.3  times  as  large  as  the  cerr espond i ng  50?  EEP. 

Tnis  ellipse  size  ratio  is  illustrated  in  Graph  1  that  follows. 

The  shape,  location,  and  orientation  cf  the  base  ellipse  do  not  affect 
the  scaling  factor.  Tlie  scaling  factor  does  depend  on  which  distribution  and 
sample  size  is  used  for  the  base  ellipse  and  converted  ellipse,  however. 
Eecause  correlation  algorithnis  assume  95?  confidence  levels  the  tables  in  the 
appendix  assume  that  the  converted  ellipse  has  this  confidence  level.  (The 
Ellipse  Radius  Patios  listed  at  the  top  cf  each  table  are  Chi-square  to 
Cni-square  and  tne  entry  in  the  table  are  F  to  F).  Eecause  of  this  95? 
convention  the  2.5  scaling  factor  for  sample  size  5  between  the  50?  F  and  93? 

F  EEFs  is  not  listed  in  the  table.  The  50?  F  to  95?  scaling  factor  (for 
sample  size  5)  of  3-3  (3.292)  is  listed  in  this  table,  however. 

FACTOES  AFFECTING  THE  SCALING  FACTOR 

A.  The  Gcnfidence  Level  of  the  Incoming  Ellipse. 

If  the  incoming  ellipse  is  based  on  a  95?  confidence  level  bound 
tnen  no  conversion  is  necessary. 

If  the  incemihg  ellipse  is  based  on  a  90?  confidence  level 
then  the  scaling  factor  will  be  bigger  than  one  and  the  resultant 
ellipse  will  be  bigger.  Examination  of  the  90?  F  radius  ratios  in 
tables  in  the  Appendix  confirms  this.  (As  do  the  conversion  factor's 
for  the  Chi-square  conversions  which  are  listed  above  the  tables.) 

If  tne  incoming  ellipse  is  based  on  a  50?  confidence  level  then 
the  scaling  factor  for  conversion  will  be  even  bigger  than  if  the 
incoming  ellipse  had  been  a  90?  confidence  ellipse.  This  can  be  seen 
botn  in  Graph  1  and  by  com.parison  of  50?  radius  ratios  with  90?  radius 
ratios  in  the  appendix. 

B.  The  Distribution  (Chi-square  or  F)  Underlying  the  EEPs. 

Chi-square  scaling  factors  are  closer  to  1  than  F  scaling 
factors,  flote  that  the  columns  of  F  cutoff  values  are  all  bigger  than 
the  corresponding  Chi-square  cut-off  value  listed  above  the  table. 

C.  The  Cample  Size. 

Sample  size  affects  the  F  scaling  factors  as  can  be  seen  in  the 
tables  in  the  appendix  and  in  Graph  The  tabl-^s  and  Graph  9  also 
illustrate  that  as  sample  size  increases  the  F  converges  toward  the 
Chi  —  sQuane . 

Sample  size  doesn't  affect  the  Cni-square  scaling  factors.  Tnis 
is  probably  why  sample  size  has  not  been  sent  to  the  correlation 
algorithm  in  the  past. 

0.  The  Degrees  of  Freedom 

This  factor  is  usually  based  on  spatial  dimension  and  hence  is 
fixed  in  most  applications  and  will  not  be  discussed  in  detail  here. 


j-  ^  .r 


III.  THE  Ay.?L:r;T  of  error  in  ellipse  size  that  can  occur 

Tnis  section  is  concerned  about  the  difference  between  what  a  person 
sitting  at  a  scope  wculd  observe  and  what  he  should  be  observing.  The 
assumption  is  that  a  conversion  has  been  done  to  make  the  ellipse  a  91'® 
ellipse  but  that  that  conversion  was  done  using  Chi-square  scaling  factors. 

If  course,  if  the  ellipses  were  based  on  the  Chi-square  distribution  no 
error  is  made.  But  what  if  the  distribution  underlying  the  incoming  ellipse 
w  pi  s  r  • 

If  the  incoming  ellipse  was  a  95*  F  there  is  still  no  error  as  the 
conversion  scale  is  1.  If,  however,  the  incoming  distribution  wasn't  at  95* 
tnen  thc^'e  is  an  error  because  a  Chi-square  radius  ratio  was  used  rather  than 
an  F  radius  ratio.  The  ratio  between  the  Chi-square  radius  ratio  and  the  F 
racius  ratio  is  the  natural  measure  of  this  error.  This  information  is 
provided  in  the  tables  in  the  appendix  under  the  unfortunate  columin  title  of 
IHI-F  RATIO  RATIO.  The  ratios  depend  on  the  confidence  level  so  there  are 
actually  two  columins.  The  entries  in  these  columns  are  measures  of  the 
conversion  error.  Graph  2  might  clarify  these  concepts.  The  inner  ellipse  in 
the  graph  is  assumed  to  be  a  sample  size  5,  2  degrees  of  freedom,  505  F 
incoming  ellipse.  Tnis  inner  ellipse  is  probably  not  actually  seen  by  the 
operator  during  correlation.  The  middle  ellipse  is  what  a  chi-square 
conversion  would  construct  and  presumably  report  to  the  operator.  The  outer 
ellipse  is  the  correctly  converted  ellipse. 

Note  that  the  difference  is  in  Graph  2  is  significant  and  examination  of 
the  tables  in  the  appendix  make  it  clear  that  even  more  significant 
differences  exist.  The  larger  RATIO  RATIO  entries  reflect  larger  errors. 

This  is  illustrated  in  Graph  3  for  which  the  90-955  based  RATIO  RATIO  is 
1.1603  and  the  50-955  based  RATIO  RATIO  is  1.5860.  This  example  and 
examination  of  the  tables  also  show  that  the  505  to  955  conversion  error  is 
always  larger  than  the  90-955  error,  as  one  would  expect.  Graph  illustrates 
how  this  error  decreases  with  sample  size.  In  the  limit,  as  sample  size 
increases,  the  RATIO  RATIO  would  approach  one  and  there  would  be  no 
significant  error. 

IV.  WHAT  DOES  THE  DIFFERENCE  BETWEEN  F  AND  CHI-SQUARE  SCALING  MEAN 

The  F  based  EEPs  are  bigger  chi-square  EEPs  because  chi-square  assumes 
that  the  amount  of  error  that  one  is  subject  to  is  known,  whereas  the  F 
assumes  the  amount  of  error  that  one  is  subject  to  is  unknown  and  only 
ceter.mined  as  data  comes  in  and  variation  is  found.  The  truth  probably  lies 
somewhere  between  these  two  assumptions.  The  fact  that  the  F  assumes  an  extra 
type  of  uncertainty  (the  amount  of  error  that  one  is  subject  to)  implies  that 
it  yields  bigger  EEPs.  The  higher  the  confidence  level  the  more  this 
uncertainty  is  reflected  and  hence  the  RATIO  RATIOS  get  bigger  as  the 
difference  between  the  base  confidence  level  and  the  resultant  increases. 

The  dependence  on  sample  size  is  more  problematical,  however.  The 
authors  of  this  report  suspect  that  unmodeled  sources  of  error  would  prevent 
tr.e  uncertainty  in  ellipse  size  from  going  away  to  the  extent  it  does  as 
sample  size  increases  using  the  F  test. 

V.  THE  IMPACT  OF  SCALING  ERRORS  ON  TESTING  FOR  COMBINATION 

The  nature  of  scaling  errors  is  that  incoming  ellipses  appear  to  be 
smaller  than  they  really  are.  This  means  that  the  incoming  ellipse  is  more 
likely  to  overlap  the  base  ellipse  by  enough  to  accept  (i.e.  there  is  less 
acceptance  the  bigger  the  RATIO  RATIO  Value.)  This  is  illustrated  in  Graph  5 
where  the  incorrectly  converted  ellipse  rejects  and  the  correctly  converted 
ellipse  accepts.  (Examples  may  be  constructed  geometrically  bearing  in  mind 
that  non- intersect i ng  ellipses  reject  and  that  if  the  center  of  one  ellipse  is 
in  the  ether  ellipse  then  the  acceptance  test  will  reject.) 
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VI.  THE  IMPACT  OF  SCALING  EHRCHS  ON  DETERMINING  A  POINT  ESTIMATE 


Tne  resultant  pcint  estir.ate  is  a  type  of  weighted  average  between  the 


two  original  point  estimates  (although  the  result  is  not  necessarily  actually 


on  the  line  segment  between  the  points.)  The  weights  are  based  on  ellipse 
"smallness."  Changing  ellipse  size  changes  the  weighting.  The  ellipse  that 
is  corrected  to  a  larger  size  is  weighted  less  and  the  location  estimate  will 
be  adjusted  away  from  the  point  estimate  corresponding  to  that  ellipse. 
Examination  of  graphs  6,  7,  and  8  in  sequence  illustrate  this  point. 


IMPACT  OF  SCALING  ERRORS  ON  DETERMINING  A  POINT  ESTIMATE 


The  resultant  ellipse  size  is  based  on  the  ellipse  sizes  of  the  two  inp. 


ellipses.  Bigger  input  yields  a  bigger  output.  See  Graphs  6,  7,  and  6  for  an 
•examrle.  See  MARC's  report,  "Testing  and  Combination  of  Confidence  Elli 
A  Geometric  Analysis",  for  more  aetails  about  the  relaticrship  of  the  ge 
of  input  ellipses  and  output  ellipses. 
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GRAPH 


o  outer  most  ellipse  is  a  correct  conversion  to 
95%  ellipse  using  r  cut-offs.  Sample  size  is  5 


Graphs  6,  7,  and  8  illustrate 
ellipse)  when  incorrectly  and 
are  combined  with  Base  Data. 
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Cni-square  vs.  F  test  CX)NFIDEIJCE  LEVEL  Conversion 
2  DECREES  OF  FREEDOM 


CHI-SC3JA-"E  cut-off  values  does  not  depend  on  the  nunt)er  of  data  points. 


5^3?  cut-off  value=  1 . 39 

90^  cut-off  value=  4.6l 

95i  cut-off  value=  5.99 

5C%  TO  95$  cut-off  ratic^  4.3C9 

93$  TO  95$  cut-off  ratio=  1.299 


ELLIPSE  RADIUS  RATIO  2.075 
nJLIPSE  RADIUS  RATIO  1.139 


CUTCr  r 
V.AL'JE 

3 

2 

1.762 
1 .656 
1.593 
1.56 
1.53^ 
1.5C^ 
1.^193 
1  .t&6 
1.^473 
1.il7 
1.452 
1 . 430 


90$ 

CJTOFF 

VALUE 

99 

13 

10.92 

8.64 

7.56 

6.92 

6.52 

6.22 

6.02 

5.84 

5.72 

5.62 

5.4 

5.18 


CUTDrr 

VALUE 

399 

33 

19.1 

13.88 

11.58 

10.28 

9.43 

8.92 

8.52 

8.2 

7.96 

7.78 

7.36 

6.93 


50$ 

CUTOFF 

RATOO 

133 

19 

10.83 

8.381 

7.246 

6.589 

6.179 

5.930' 

5.687 

5.518 

5.385 

5.292 

5.063 

4.860 


CUTOFF 

RAHO 

4.030 

2.111 

1.749 

1.606 

1.531 

1.485 

1.453 

1.434 

1.415 

1.404 

1.391 

1.384 

1.362 

1.347 


50$ 

RADIUS 

RAHO 

11.53 

4.358 

3.292 

2.895 

2.691 

2.567 

2.485 

2.435 

2.384 

2.349 

2.320 

2.300 

2.251 

2.204 


90$ 

RADIUS 

RAHO 

2.007 

1.452 

1.322 

1.267 

1.237 

1.218 

1.205 

1.197 

1.189 

1.184 

1.179 
1.176 
1.167 
!  1.160 


50  CC-F 
RATIO 
RATOO 

5.5556 
2.0993 
1.5860, 
1.3946 
1.2963 
1.2365 
1.1975 
1.1731 
1.1483 
1.1316 
1.1179 
1.1082 
1.0845 
1 .0620 


90  CRI-F 
PwATIO 
FAvTOO 
1.7614 
1.2748 

1.1603 
1.1120 
1.0353 
1.0693 
1.0579 
1 .0507 
1.0437 
1.0396 
1.0350 

1.0323 

1.0243 

1.0184 
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Gii-sqjare  vs.  F  test  LET^/EL  Conversion 

3  DEGREES  OF  FRrlRlXJ-i 


CtI-SQUA?£  cot-off  valiies  does  not  depend  on  the  nurrDer  of  data  points. 


53^  cut-off  value =  2.37 
93 i  cut-off  \alue=  6.25 
95^  cut-off  value=  7.81 

53i  TO  95?  cut-off  ratio  3.295  ELLIPSE  RADIUS  RAnO=  1.815 
93?  TO  95?  cut-off  ratio=  1.2^9  ELLIPSE  RADIUS  RmO=  1.117 


50? 

90? 

95? 

50? 

90? 

50? 

90? 

50  CC-F 

90  C-L-F 

SA'-?LE 

'^>J  iL/*  *. 

CL  ior  r 

CUTO.=7 

CUTOFF 

CUTOFF 

RADIUS 

RADIUS 

RATIO 

PuATIO 

SIZE 

v;lue 

VAL'JE 

VALUE 

R4TT0 

RATIO 

RAHO 

RATIO 

RAHO 

RATIO 

3 

5.13 

160.8 

643 

126.3 

4.029 

11.23 

2.007 

6.1915 

1.7962 

3.39 

27.48 

57.6 

16.99 

2.096 

4.122 

1.447 

2.2708 

1.2954 

5 

3 

16.17 

27.84 

9.28 

1.721 

3.046 

1.312 

1.6782 

1.1740 

6 

2.823 

12.57 

19.77 

7.003 

1.572 

2.646 

1.254 

1.4578 

1.1221 

7 

2.721 

10.86 

16.23 

5.964 

1.494 

2.442 

1.222 

1.3454 

1 .0933 

8 

2.653 

9.87 

14.23 

5.372 

1.446 

2.317 

1.202 

1 .2769 

1.0762 

9 

2.6(3 

9.21 

13.05 

4.994 

1.416 

2.234 

1.190 

1.2311 

1.0651 

13 

2.53 

8.76 

12.21 

4.732 

1.393 

2.175 

1.180 

1.1934 

1.0553 

11 

2.555 

8.43 

11.58 

4.530 

1.373 

2.128 

1.172 

1.1725 

1.0487 

12 

2.535 

8.19 

11.13 

4.390 

1.358 

2.095 

1.165 

1.1543 

1.0430 

13 

2.52 

7.93 

10.77 

4.273 

1.349 

2.067 

1.161 

1.1383 

1.0395 

1^1 

2.505 

7.83 

10.47 

4.179 

1.337 

2.044 

1.156 

1.1262 

1.0346 

17 

2.^178 

7.47 

9.87 

3.983 

1.321 

1.995 

1.149 

1.0994 

1.0285 

22 

2.1148 

7.14 

9.3 

3.799 

1.302 

1.949 

1.141 

1.0737 

1.0212 

E; 

c, 

f 


